LIMITED  INVESTIGATION  OF  AIR-TO-AIR 
LASER  DETECTION  AND  RANGING 

(PROJECT  WALKER  RANGER) 


A 

F 

F 

T 

C 


BRIAN  T.  DEAS 
Major,  USAF 
Project  Test  Pilot  / 
Project  Manager 

RYAN  A.  HOWLAND 
Captain,  USAF 
Project  Test  Pilot 


KRISTOFERA.  PETERSON 
YD-02,  USAF 

Project  Flight  Test  Engineer 


REID  A.  LARSON 

Captain,  USAF 

Project  Flight  Test  Engineer 


SARAH  K.  HELMS 

Captain,  USAF 

Project  Flight  Test  Engineer 


December  2009 

FINAL  TECHNICAL  INFORMATION  MEMORANDUM 


Approved  for  public  release;  distribution  is  unlimited. 


AIR  FORCE  FLIGHT  TEST  CENTER 
EDWARDS  AIR  FORCE  BASE,  CALIFORNIA 
AIR  FORCE  MATERIAL  COMMAND 
UNITED  STATES  AIR  FORCE 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


This  Technical  Information  Memorandum  (AFFTC-TIM-09-1 1,  Limited  Investigation 
of  Air-to-Air  Laser  Detection  and  Ranging,  Project  Walker  Ranger)  was  submitted  under  Job 
Order  Number  MT09A700  by  the  Commandant,  USAF  Test  Pilot  School,  Edwards  AFB, 
California  93524-6485. 


Prepared  by: 


Reviewed  by: 


BRIAN  T.  DEAS 


Major,  USAF 
Project  Manager 


RYAM  A.  HOWLAND 
Captain,  USAF 
Project  Test  Pilot 


KRISTOFER  A.  PETERSON 
YD-02,  USAF 


Project  Flight  Test  Engineer 


REID  A.  LARSON 

Captain,  USAF 

Project  Flight  Test  Engineer 


SArahk.  heLms^ 


Captain,  USAF 

Project  Flight  Test  Engineer 


a 


/iMo' 


TIMOTHYK.  McDonald 
YA-02,  USAF 
TPS  Staff  Advisor 


MARY  E.  McNEELY 
YD-02,  USAF 

Chief,  Test  Management  Branch 


DAVID  L.  VANHOY 
YD-03,  USAF  (  / 

Technical  Director/ USAF  Test  Pilot  School 


This  report  has  been  approved  for 
publication: 


JAN  1 


9  2011 


u 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining 

the  data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for 
reducing  this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington, 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not 
display  a  currently  valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1.  REPORT  DATE  (DD-MM-YYYY) 

11-12-2009 

2.  REPORT  TYPE 

Final  Technical  Information  Memorandum 

3.  DATES  COVERED  (From  -  To) 

1  May  2009  to  15  Nov  2009 

4.  TITLE  AND  SUBTITLE 

Limited  Investigation  of  Air-to-Air  Laser  Detection  and  Ranging 
(Project  Walker  Ranger) 

5a.  CONTRACT  NUMBER 

F3361 5-00-C-1 709 

5b.  GRANT  NUMBER 

N/A 

5c.  PROGRAM  ELEMENT  NUMBER 

N/A 

6.  AUTHOR(S) 

Deas,  Brian,  Major,  USAF 

Howland,  Ryan,  Captain,  USAF 

Peterson,  Kristofer,  YD-02,  USAF 

Larson,  Reid,  Captain,  USAF 

Helms,  Sarah,  Captain,  USAF 

5d.  PROJECT  NUMBER 

MT09A700 

5e.  TASK  NUMBER 

N/A 

5f.  WORK  UNIT  NUMBER 

N/A 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

AND  ADDRESS(ES) 

Air  Force  Flight  Test  Center 

412th  Test  Wing 

USAF  Test  Pilot  School 

220  South  Wolfe  Ave 

Edwards  AFB  CA  93524-6485 

8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 

AFFTC-TIM-09-1 1 

9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Attn:  Stephen  Cain 

AFIT/ENG 

2950  Hobson  Way 

Wright  Patterson  AFB.  OH  45433-7765 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

Stephen  Cain 

11.  SPONSOR/MONITOR’S  REPORT 

number(S)  AFFTC-TIM-09-1 1 

12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

13.  SUPPLEMENTARY  NOTES 

CA:  Air  Force  Flight  Test  Center  Edwards  AFB  CA  CC:  012100 

14.  ABSTRACT 

This  report  presents  the  results  of  the  Limited  Investigation  of  Air-to-Air  Laser  Detection  and  Ranging  (Project  Walker  Ranger),  a 
demonstration  of  the  Advanced  Scientific  Concepts  LADAR  capabilities.  The  overall  test  objective  of  the  Walker  Ranger  TMP  was  to 
demonstrate  the  ability  of  the  ASC  LADAR  to  gather  an  accurately  ranged  three  dimensional  image  of  an  air-to-air  target.  Due  to 
LADAR  hardware  issues/late  delivery  of  a  flight-qualified  test  item  testing  was  limited  to  a  single,  partially  successful,  ground  test.  The 
Air  Force  Institute  of  Technology  requested  this  testing.The  USAF  TPS,  Class  09 A,  conducted  this  testing  on  22  September  2009.  No 
test  objectives  were  met. 

15.  SUBJECT  TERMS 

Walker  Ranger,  LADAR,  flight  test,  RASCAL 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION 
OF  ABSTRACT 

SAME  AS 
REPORT 

18.  NUMBER 
OF  PAGES 

37 

19a.  NAME  OF  RESPONSIBLE  PERSON 

Stephen  Cain 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

UNCLASSIFIED  UNCLASSIFIED  UNCLASSIFIED 

19b.  TELEPHONE  NUMBER  (include  area 
code) 

937-255-3636x4716 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


111 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


This  page  intentionally  left  blank. 


IV 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


ACKNOWLEDGEMENTS 

The  Walker  Ranger  test  team  would  like  to  thank  the  various  personnel  at  the  USAF  Test 
Pilot  School  (TPS)  who  made  this  multi-disciplinary  technical  effort  possible  within  the 
relatively  short  time  allotted  for  testing.  Thank  you  to  Santos  Escobar  and  Lionel  Banuelos  for 
putting  in  countless  hours  troubleshooting  and  ensuring  the  pod  was  ready  for  test.  Thanks  to  Lt. 
Mildred  Ramos,  Lt.  Angel  Serna  and  Lt.  Hoa  Nguyen,  without  whom  this  project  would  not  have 
succeeded.  Their  dedication  was  the  defining  contribution  that  allowed  this  test  to  yield  valuable 
results. 


v 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


This  page  intentionally  left  blank. 


vi 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


EXECUTIVE  SUMMARY 

The  US  Air  Force  Test  Pilot  School  (TPS)  class  09A  Walker  Ranger  test  management  project 
(TMP)  group  accomplished  ground  testing  of  the  Advanced  Scientific  Concepts  (ASC)  laser 
detection  and  ranging  (LADAR)  system.  The  planned  test  consisted  of  a  ground  test  and  three 
flight  test  sorties.  However,  flight  testing  was  not  completed.  Due  to  LADAR  hardware  issues, 
late  delivery  of  a  flight-qualified  test  item  the  TMP  group  was  limited  to  a  single,  partially 
successful,  ground  test.  The  ground  test  succeeded  in  gathering  LADAR  determined  ranges  to 
compare  to  known  ground  target  ranges.  However  the  ground  test  did  not  succeed  in  gathering 
sufficient  DGPS  data  for  comparison. 

The  overall  test  objective  of  the  Walker  Ranger  TMP  was  to  demonstrate  the  ability  of  the  ASC 
LADAR  to  gather  an  accurately  ranged  three  dimensional  image  of  an  air-to-air  target. 

Specific  test  objectives  are  shown  below: 

1 .  Determine  the  range  accuracy  of  the  ASC  LADAR  using  differential  GPS  (DGPS)  as 
the  truth  position  source 

2.  Determine  a  maximum  target  detection  range  against  an  airborne  target 
No  test  objectives  were  met. 

The  TMP  ground  test  occurred  on  22-23  September  2009.  The  ground  test  was  performed  on  the 
TPS  ramp  and  involved  ground  tape  measurements  along  with  DGPS  and  LADAR  ranging 
between  the  test  and  target  F-16s.  An  Air  Force  Flight  Test  Center  (AFFTC)  F-16D  tail  number 
90-00797  was  used  as  the  test  aircraft  and  F-16D  tail  number  87-00391  was  used  as  a  support 
target  aircraft.  The  three  planned  flight  tests  were  not  completed. 

The  test  configuration  had  the  LADAR  internally  loaded  (LIL)  on  the  Reconfigurable  Airborne 
Sensor,  Communication  and  Laser  (RASCAL)  pod.  The  LIL  RASCAL  was  loaded  on  weapons 
station  three  of  the  F-16D  test  aircraft.  Additionally,  reflective  tape  was  applied  externally  to 
several  locations  on  the  target  aircraft.  Both  F-16Ds  were  equipped  with  GPS  aided  inertial 
navigation  reference  (GAINR-LITE)  systems  designed  to  provide  the  required  data  for  DGPS 
processing  of  range  between  the  two  aircraft  and  to  serve  as  the  test  truth  source.  However,  the 
test  aircraft  GAINR-LITE  system  failed  to  collect  data  during  the  ground  test;  therefore,  ground 
measurements  were  used  as  truth  data  instead  of  DGPS  processed  range  information. 

The  LADAR  generated  ranged  images  by  sending  out  pulses  of  1.57  micrometer  wavelength 
diffused  laser  light.  Reflections  of  the  laser  pulses  off  targets  within  a  nine  degree  field  of  view 
were  captured  optically  and  directed  onto  a  128x128  pixel  focal  plane  array.  Each  pixel  recorded 
reflected  intensity  to  generate  an  image,  while  also  recording  round  trip  time  of  flight  of  the 
pulse  to  generate  target  range. 

During  the  ground  test,  signal  returns  from  the  target  F-16  at  300  feet  were  not  sufficient  to 
generate  a  clear  three  dimensional  image.  Additionally,  LADAR  range  measurements  did  not 
achieve  the  expected  accuracy  of  three  inches.  Major  contributors  to  inaccurate  range 
measurements  were  return  signal  measurement,  return  signal  storage,  and  system  calibration 
variability. 
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INTRODUCTION 

This  Technical  Information  Memorandum  reports  on  the  test  management  project  (TMP)  that 
sought  to  investigate  the  air-to-air  ranging  accuracy  and  maximum  range  capability  of  the  laser 
detection  and  ranging  (LADAR)  system  developed  by  Advanced  Scientific  Concepts  (ASC). 
The  system  was  a  next  generation  sensor  with  a  wide  range  of  potential  uses,  one  of  which  could 
be  assisting  with  air-to-air  refueling  (AAR)  of  unmanned  aerial  vehicles  (UAV).  This  project 
also  investigated  a  ranging  algorithm,  developed  by  the  test  team,  for  post-processing  the 
LADAR  data  to  determine  if  the  new  algorithm  provided  improvement  in  range  resolution. 

The  Responsible  Test  Organization  was  the  412th  Test  Wing.  Testing  was  conducted  under  the 
USAF  Test  Pilot  School  (TPS)  Job  Order  Number  MT09A700  and  was  accomplished  22-23 
September  2009.  The  Walker  Ranger  TMP  was  conducted  at  the  request  of  the  United  States  Air 
Force  Test  Pilot  School  in  conjunction  with  the  Air  Force  Institute  of  Technology  Department  of 
Electrical  Engineering.  The  testing  was  conducted  with  the  LADAR  internally  loaded  (LIL)  on 
the  Reconfigurable  Airborne  Sensor,  Communication  and  Laser  (RASCAL)  pod  and  loaded  on 
F-16D  test  aircraft,  tail  number  90-00797  (figure  1).  Another  F-16D,  tail  number  87-00391,  was 
used  as  a  target  for  ranging.  Flight  testing  was  not  accomplished. 


Figure  1 :  LIL  RASCAL  Pod  on  Test  Aircraft  during  Ground  Test 


Background 

The  LADAR  system  was  evaluated  for  a  wide  range  of  military  uses  because  of  its  unique 
capability  to  simultaneously  image  and  range  a  target  scene  providing  a  three  dimensional 
representation  at  video  frame  rates.  Two  of  the  potential  uses  of  the  system  were  three 
dimensional  target  identification  and  autonomous  system  guidance.  The  autonomous  system 
guidance  mission  provided  the  basis  for  all  testing  performed  under  the  Walker  Ranger  TMP. 

A  LADAR  system,  such  as  the  ASC  system  used  for  this  test,  may  have  been  able  to  provide 
guidance  to  a  UAV  during  AAR  operations.  Autonomous  UAV  close  formation  in  the  AAR 
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position  had  been  accomplished  using  differential  global  positioning  systems  (DGPS)  to  provide 
relative  position  with  an  average  spherical  error  of  1 .2  inches  between  the  tanker  and  the  receiver 
aircraft  (reference  1).  However,  a  non-GPS  solution  was  desired  to  maintain  autonomous 
capability  in  a  GPS  denied  environment. 

Target  ranging  against  a  known-range  target  had  not  been  accomplished  with  this  system  prior  to 
this  TMP.  Range  accuracy  of  the  LADAR  was  determined  by  ASC  to  be  approximately  three 
inches;  however  this  determination  was  made  in  a  laboratory  using  fiber  optics.  Additional 
investigation  was  done  to  evaluate  the  relative  ranging  errors  of  the  system.  For  the  purposes  of 
this  test,  the  term  relative  range  referred  to  the  range  measured  between  two  objects  in  the  target 
scene.  Relative  range  was  different  from  absolute  range  which  was  defined  for  this  test  to  be  the 
range  from  any  point  in  the  target  scene  to  the  LADAR  receiver  itself.  Open  air  range 
comparisons  between  LADAR  generated  ranges  and  accurately  ranged  truth  sources  had  not  yet 
been  evaluated. 

Test  Item  Description 
Physical  Description 

The  ASC  LADAR  under  evaluation  for  this  test  consisted  of  four  major  components:  a  laser 
transmitter,  a  laser  receiver  focal  plane  array  with  interchangeable  optics,  a  visible  spectrum 
television  (TV)  camera,  and  a  computer  loaded  with  appropriate  software  to  operate  the  system. 
The  uninstalled  system  was  six  inches  wide  by  eight  inches  tall  by  eleven  inches  deep  and  is 
shown  in  figure  2  (reference  2). 


Figure  2:  Uninstalled  LADAR  System  (reference  2) 
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Figure  3:  LADAR  Internally  Loaded  RASCAL  (LIL  RASCAL) 

(reference  3) 


The  laser  transmitter  generated  pulses  of  1.57  pm  light  at  a  peak  power  of  18  mJ  (reference  2). 
The  pulses  were  then  diffused  to  ensure  they  covered  the  field  of  view  (FOV)  of  the  receiver. 
The  receiver  optics  were  designed  to  be  interchangeable,  but  an  85  mm  lens  with  a  nine  degree 
FOV  was  the  sole  setup  for  this  test.  The  TV  camera  provided  a  secondary  visual  reference  for 
the  target  scene  and  had  a  FOV  of  approximately  twenty  degrees. 

The  LADAR  components  were  mounted  in  the  RASCAL  pod  with  a  down  and  aft  look  angle 
sixty  degrees  offset  from  horizontal.  The  LADAR  internally  loaded  RASCAL  (LIL  RASCAL) 
configuration  was  designed  to  withstand  the  RASCAL  flight  environment  to  include 
acceleration,  vibration,  and  temperature.  The  RASCAL  flight  environment  was  established  by 
the  TPS  SENIOR  RASCAL  test  program  and  is  documented  in  the  Certification  Data  Package, 
RASCAL  Pod-7  (reference  4).  The  LIL  RASCAL  assembly  is  shown  in  figure  3. 

In  addition  to  the  ASC  LADAR  software,  the  Walker  Ranger  team  developed  a  normalized 
variable  shaping  (NOVAS)  range  estimation  algorithm  to  post-process  the  data.  Previous 
LADAR  algorithms  used  straight  correlation  with  a  predetermined  or  optimized  pulse  shape. 
The  NOVAS  estimator  allowed  the  correlation  reference  pulse  to  vary  in  shape  and  symmetry  to 
best  match  the  shape  of  the  laser  pulse  received  by  the  detector.  The  intent  of  NOVAS  was  to 
compensate  for  error  due  to  various  physical  interactions  within  the  laser  cavity  as  well  as  with 
the  target  shape.  A  full  description  of  the  NOVAS  algorithm  is  contained  in  appendix  B. 

System  Operation 

For  each  pulse  sent  by  the  LADAR,  an  image  was  created  of  all  objects  in  the  FOV.  Each  pixel 
in  the  array  recorded  the  intensity  and  time  history  of  the  laser  energy  reflected  by  the  target. 
The  intensity  values  built  the  image  scene  while  the  time  history  built  the  range  to  the  target 
based  on  the  round  trip  time  of  flight  of  the  laser  pulse. 
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The  camera  operated  in  two  modes:  trigger  mode  and  staring  underwater  laser  ranging  (SULAR) 
mode.  Trigger  mode  was  the  normal  operating  mode  while  SULAR  mode  was  designed  to 
overcome  target  obscuration  from  semi-transparent  substances  such  as  water,  smoke,  or  haze. 
The  difference  was  the  method  of  commanding  return  pulse  data  to  be  saved.  After  a  laser  pulse 
was  sent  out  in  trigger  mode,  each  pixel  waited  to  record  data  until  a  certain  user  defined 
threshold  of  energy  was  received.  Once  the  threshold  was  crossed,  the  pixel  saved  a  rough 
distance  to  the  target  as  well  as  20  intensity  values  separated  by  approximately  2.5  ns  to  capture 
the  target  return  pulse  waveform.  Each  pixel  operated  independently  until  a  user  defined  time 
had  elapsed,  at  which  point  pixels  that  had  not  received  a  target  return  pulse  recorded  a  default 
value.  In  SULAR  mode,  all  pixels  in  the  array  recorded  data  at  the  same  time,  starting  at  a  user 
defined  time  interval.  Only  returns  from  targets  inside  this  range  “gate”  were  recorded  by  the 
LADAR.  Pixels  without  a  target  in  the  range  gate  recorded  background  radiation  only  and  no 
actual  laser  energy.  A  more  detailed  description  of  these  modes  is  contained  in  appendix  B. 

Test  Objectives  and  Limitations 

The  overall  test  objective  of  the  Walker  Ranger  TMP  was  to  demonstrate  the  ability  of  the  ASC 
LADAR  to  gather  an  accurately  ranged  three  dimensional  image  of  an  air-to-air  target. 

Specific  test  objectives  are  shown  below: 

1 .  Determine  the  range  accuracy  of  the  ASC  LADAR  using  differential  GPS  (DGPS)  as 
the  truth  position  source 

2.  Determine  a  maximum  target  detection  range  against  an  airborne  target 
Neither  test  objective  was  accomplished. 

Originally,  the  LADAR  was  scheduled  to  be  delivered  by  the  contractor  on  1  August  2009. 
Significant  technical  challenges  and  sub-contractor  scheduling  led  the  contractor  to  slip  the 
delivery  date  of  the  LIL  RASCAL  pod  numerous  times.  The  system  suffered  several  delays  due 
to  technical  issues  with  system  hardware  and  design  compatibility.  These  technical  delays 
forced  the  contractor  to  miss  the  scheduled  vibration  test,  a  prerequisite  for  flight  certification, 
and  ultimately  prevented  flight  test  of  the  LADAR  within  the  test  window. 

TPS  student  scheduling  requirements,  vibration  test  availability,  and  TMP  deadlines  allowed  a 
minimal  24  hour  period  to  accomplish  the  ground  test.  This  forced  the  test  team  to  accept  a 
rushed  acceptance  test,  ground  check,  electro-magnetic  interference  and  compatibility  test 
(EMI/C),  and  ground  data  timetable.  Additionally,  the  LIL  RASCAL  used  for  the  ground  test 
had  not  been  cleared  for  flight. 

Separate  ground  checks  and  EMI/C  that  were  planned  to  be  accomplished  prior  to  ground  test 
data  collection  were  eliminated  to  prioritize  data  collection.  This  required  the  test  team  to 
troubleshoot  system  problems  while  collecting  ground  data.  During  an  eight  hour  ground  test 
window,  five  hours  were  spent  investigating  system  problems  that  included  no  cockpit  multi- 
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function  display  (MFD)  video,  no  ability  to  control  the  LADAR  from  the  cockpit  via  the  ECM 
C-9492,  and  no  ability  to  control  RASCAL  from  the  OQO™  palmtop  computer. 

Ultimately,  the  test  program  was  only  able  to  collect  three  hours  of  ground  data  from  a  non-flight 
worthy  LIL  RASCAL  pod  instead  of  three  test  flights. 
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TEST  AND  EVALUATION 
Test  Execution 

The  test  aircraft  (F-16D  90-00797)  and  target  aircraft  (F-16D  87-00391)  were  parked  on  the  TPS 
ramp  separated  by  a  distance  of  335  feet.  Both  aircraft  used  ground  power,  and  GPS  aided 
inertial  navigation  reference  (GAINR-LITE)  systems  were  used  to  collect  DGPS  distance  data. 
DGPS  data  were  collected  for  the  required  duration  of  one  hour  for  the  target  aircraft  only. 
DGPS  data  for  the  test  aircraft  were  only  gathered  for  90  seconds  due  to  a  system  malfunction. 
The  original  intent  for  the  DGPS  data  was  to  determine  the  mean  and  standard  deviation  of 
DGPS  range  between  the  GAINR-LITE  systems  to  compare  against  the  412  TW/ENR  stated 
position  accuracy  for  the  GAINR-LITE  of  1.5  feet.  Insufficient  data  were  collected  to  address 
potential  drift  of  the  system  positions.  Further  ground  testing  will  be  required  prior  to  flight  test 
to  ensure  accurate  range  results  for  comparison  to  LADAR  flight  test  ranging.  Verify  mean  and 
standard  deviation  of  GAINR-LITE  meet  predictions  from  412  TW/ENR  prior  to  flight  test 
of  the  LIL  RASCAL.  (Rl) 1 

The  LADAR  system  was  installed  in  the  RASCAL  pod  and  the  pod  was  mounted  on  station  3  of 
the  test  aircraft.  The  LADAR  system  was  configured  in  the  pod  to  image  targets  at  a  60  degree 
angle  aft  and  downward  of  the  pod  through  a  polycarbonate  viewing  window.  For  the  ground 
test,  an  adjustable  mirror  was  positioned  below  the  RASCAL  pod  viewing  window  such  that 
laser  energy  pulses  were  projected  parallel  to  the  ground  to  view  targets  in  front  of  the  test 
aircraft.  Figure  4  illustrates  the  RASCAL  pod,  LADAR  system,  and  adjustable  mirror  setup 
used  to  complete  the  tests. 
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Optical 
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Adjustable 
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Figure  4:  Ground  test  setup  with  LIL  RASCAL  and  adjustable  ground  test  mirror 

A  series  of  reflective  targets  were  then  placed  in  front  of  the  test  aircraft  at  known  positions  and 
ranges  along  the  ground.  The  reflective  targets  consisted  of  small  wood  blocks  with  dowel  rods 
protruding  from  the  blocks  and  a  small  reflective  flag  attached  to  the  end  of  the  dowel.  The 
reflective  flags  were  sized  to  minimize  saturation  of  individual  image  pixels  for  later  post- 


1  Numerals  preceded  by  an  R  within  parentheses  at  the  end  of  a  sentence  correspond  to  the  recommendation 
numbers  tabulated  in  the  Conclusions  and  Recommendations  section  of  this  report.  (The  Author’s  Guide  to  Writing 
Air  Force  Flight  Test  Center  Technical  Reports,  reference  5) 
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processing.  Figure  5  shows  a  schematic  of  the  reflective  flag  layout  placed  in  front  of  the  test 
aircraft  for  imaging.  The  layout  was  designed  such  that  the  reflectors  would  appear  evenly 
spaced  on  a  horizontal  line  in  the  LADAR  image.  The  target  aircraft  was  located  on  the  center 
line  near  the  300  foot  target  but  is  not  shown  in  figure  5  for  clarity.  Also  not  shown  in  figure  5 
are  targets  at  201,  202,  203,  and  204  feet  in  the  vicinity  of  the  200  foot  target.  Figure  6  shows  a 
photograph  of  the  test  aircraft  viewing  the  reflector  flag  array  and  the  target  aircraft  used  for 
LADAR  range  ground  tests.  Figure  7  shows  a  typical  LADAR  image  of  the  target  array 
captured  during  the  ground  test.  See  appendix  A  for  additional  similar  images. 

!'  500’  “I 


Reflective  Flaas 


Figure  5:  Ground  test  setup  depicting  reflective  target  layout  (for  clarity, 
the  target  aircraft  is  not  shown  in  position  near  the  300  foot  target) 


Figure  6:  Photograph  of  the  ground  test  setup 


7 


Project  Walker  Ranger 
December  2009 


Edwards  Air  Force  Base 
Air  Force  Flight  Test  Center 


Figure  7:  Typical  LADAR  image  captured  during  the  ground  test 


Results  and  Analysis 

Results  from  the  ground  test  are  broken  down  into  five  separate  topics  for  discussion.  The  first 
section  addresses  the  overall  system  integration  of  the  LIL  RASCAL.  The  second  section 
addresses  the  precision  and  accuracy  of  the  range  results  reported  by  the  LADAR  to  provide  a 
statistical  basis  for  further  analysis.  The  third  section  addresses  the  effects  of  timing  system 
variability  on  the  range  measurements.  The  fourth  section  focuses  on  the  compatibility  of  the 
LADAR  output  and  the  signal  processing  methods  used  by  the  NOVAS  algorithm.  The  fifth  and 
final  section  discusses  some  aspects  of  the  total  system  performance  in  relation  to  utility  for 
future  flight  research,  development,  test,  and  evaluation. 

System  Integration 

The  LIL  RASCAL  was  not  well  integrated  with  the  test  aircraft  during  the  ground  test.  The  LIL 
RASCAL  failed  to  display  video  on  the  F-16  MFD  during  all  system  operations.  The  cockpit 
operation  with  the  ECM  C-9492  and  OQO™  palmtop  computer  were  ineffective  for  LIL 
RASCAL  control. 

When  the  LIL  RASCAL  was  mated  to  the  F-16D,  the  LADAR  video  source  was  connected  to  an 
AGM-65  Maverick  video  input  so  that  LIL  RASCAL  images  could  be  assessed  in  real  time  on 
the  cockpit  MFD.  An  incompatibility  between  the  signal  output  from  the  LIL  RASCAL  and  that 
required  by  the  aircraft  prevented  video  from  being  displayed  on  the  F-16  MFD,  making  it 
impossible  for  the  test  team  to  monitor  the  system  from  the  cockpit.  During  the  ground  test,  an 
external  monitor,  keyboard,  and  mouse  were  attached  directly  to  the  RASCAL  to  monitor  and 
control  the  LADAR  system.  This  solution  worked  on  the  ground  but  was  unusable  for  flight. 
Additionally,  the  OQO™  screen  resolution  was  too  low  and  the  screen  size  was  too  small  to 
provide  useable  airborne  feedback.  Feedback  to  the  pilot  of  LIL  RASCAL  operation  was 
unsatisfactory.  Provide  video  from  the  LIL  RASCAL  to  the  F-16  MFD  to  allow  assessment 
in  real  time  of  LADAR  operation.  (R2) 
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The  pilot  controlled  the  RASCAL  through  the  ECM  C-9492  control  panel  (figure  8).  This 
control  unit  was  poorly  suited  for  controlling  the  LIL  RASCAL  due  to  the  button  design  that 
latched  in  the  down  position  when  the  button  was  pressed.  This  led  to  difficulties  when 
commanding  the  LADAR  because  system  action  only  began  after  the  retum/up  stroke  of  the 
button.  If  a  button  was  pressed  and  left  latched  in  the  down  position,  no  system  response  was 
initiated.  Even  if  the  button  was  later  pressed  to  return  to  the  up  position,  the  RASCAL  would 
only  occasionally  perform  the  desired  action.  The  C-9492  provided  limited  system  feedback 
through  lights  on  the  panel;  however  the  lights  only  illuminated  momentarily.  These  indications 
were  missed  by  the  pilot  on  several  occasions  because  of  sun  glare,  a  finger  remaining  on  the 
button,  or  simply  not  looking  at  the  right  indicator  light  at  the  right  time. 

AFFTC  Modification  Operational  Supplement  (MOS)  09-01  was  written  specifically  for 
operation  of  the  RASCAL  pod  and  installed  LADAR  system  from  the  front  cockpit  of  the  F-16D 
test  aircraft.  MOS  09-01  focused  on  the  operation  of  the  ECM  C-9492  panel  and  viewing  the 
LADAR  returns  (using  AGM-65  video  feed)  on  the  front  cockpit  MFD.  The  MOS  was 
originally  written  based  on  inputs  from  both  in-house  engineers  at  Test  Pilot  School  and  the 
prime  contractor  (ASC)  who  had  worked  together  to  integrate  the  RASCAL  pod  and  LADAR 
system  to  the  F-16D  systems.  The  MOS,  when  used  during  ground  test,  proved  not  to  be  useful 
and  was  unsatisfactory  for  use  in  any  flight  test.  The  deficiencies  with  the  MOS  were  primarily 
due  to  two  inter-related  factors:  (1)  the  ECM  C-9492  panel  did  not  physically  operate  in  the 
manner  originally  desired  to  provide  necessary  feedback  to  the  pilot  with  respect  to  LADAR 
system  operation  and  (2)  the  language  in  the  MOS  did  not  accurately  reflect  the  actual  operation 
of  the  RASCAL  pod  and  LADAR  system  via  the  ECM  C-9492  panel.  Several  RASCAL  pod 
systems  and  LADAR  functions  were  intended  to  be  controlled  by  either  pressing/releasing  or 
pressing/holding/releasing  buttons  on  the  ECM  C-9492  panel.  For  example,  the  LADAR 
initialization  function  used  prior  to  imaging  targets  was  intended  to  be  controlled  by  pressing  the 
appropriate  button  on  the  ECM  panel  for  the  initialization,  holding  the  button  for  three  seconds, 
then  releasing  the  button.  This  would  allow  an  internal  circuit  to  be  closed  for  three  seconds 
while  the  button  was  held  pressed  down  and  allow  adequate  time  for  the  LADAR  system  to 
complete  its  initialization  process.  However,  once  the  button  was  released,  the  circuit  would  be 
broken  and  the  initialization  process  would  discontinue.  The  ECM  C-9492  panel  was  not 
mechanized  in  this  fashion,  however.  In  fact,  once  a  button  on  the  panel  was  pressed  down,  it 
remained  locked  in  the  down  position  until  the  button  was  subsequently  pressed  a  second  time  to 
release  it  from  the  down  position.  Further,  it  was  intended  that  the  panel  would  provide  some 
visual  feedback  that  something  had  occurred  in  the  system  (such  as  a  light  illuminating  when  the 
circuit  was  closed).  No  visual  feedback  to  the  pilot  in  the  front  cockpit  occurred;  neither  the 
ECM  C-9492  panel  nor  the  MFD  gave  any  indication  of  any  system  actuation  during  ground  test. 
These  facts  were  neither  known  nor  considered  prior  to  the  original  writing  of  the  MOS  and 
ground  testing,  causing  a  large  amount  of  confusion  during  test  execution.  The  compressed 
schedule  that  forced  system  integration  testing  and  ground  testing  to  be  performed 
simultaneously  did  not  allow  adequate  time  to  correct  the  problem.  The  MOS  did  not  provide 
adequate  information  to  the  pilot  and  could  not  be  used  in  the  flight  test  environment  in  its 
current  form.  Cockpit  control  of  the  LIL  RASCAL  was  not  well  defined,  not  intuitive  and 
unsatisfactory.  Provide  an  alternate  interface  or  control  panel  and  accurate  written 
instructions  to  operate  the  LADAR  system  in  the  RASCAL  pod.  (R3) 
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Figure  8:  ECM  control  panel  C-9492;  left  console 
(from  T.O.  1F-16CM-34-1-1,  reference  6) 


Accuracy  and  Precision  of  Reported  Range  Results 

The  LADAR  did  not  achieve  expected  range  accuracy  against  stationary  ground  targets.  Based 
on  published  documents  from  the  contractor,  the  reported  ranges  from  the  LADAR  were 
expected  to  be  within  three  inches  of  the  actual  range  (reference  2).  The  standard  deviation  of 
range  measurements  taken  during  the  ground  test  indicated  the  expected  range  accuracy  was  not 
met. 

A  summary  of  LADAR  generated  ranges  collected  during  the  ground  test  is  shown  in  table  1 . 
Some  of  the  ranges  were  collected  against  the  prepositioned  reflective  targets  of  known  range, 
while  others  were  targets  of  opportunity  of  unknown  range.  Targets  that  appear  more  than  once 
in  the  table  indicate  a  different  algorithm  was  used  to  estimate  the  range  to  the  same  target.  Each 
line  in  the  tables  represents  a  single  pixel  used  to  generate  the  range  estimation.  The  same  target 
returns  captured  by  the  LADAR  were  analyzed  by  two  different  algorithms  for  estimating  range. 
The  range  estimations  in  table  1  were  generated  with  the  NOVAS  algorithm  and  the  range 
estimation  algorithm  internal  to  the  contractor  provided  LADAR  control  software.  The  ASC 
algorithm  was  similar  to  the  NOVAS  algorithm  in  concept,  but  was  different  in  implementation 
as  described  in  appendix  B. 

A  sample  of  thirty  images  was  used  to  generate  thirty  range  estimations  for  each  of  the  targets 
shown  in  table  1  using  both  the  NOVAS  algorithm  and  the  ASC  algorithm.  The  sample  mean 
and  standard  deviation  for  each  set  of  target  measurements  is  provided  in  the  table.  A 
confidence  interval  for  the  LADAR  system’s  population  mean  of  range  to  each  target  is  provided 
for  95  percent  certainty.  Recall  the  confidence  interval  is  given  by  the  equation 

*±t»-i(«/2)4= 

yjn 

where  X  is  the  sample  mean,  s  is  the  sample  standard  deviation,  n  is  the  number  of  data  points  in 
the  sample,  t„.i{a/2)  is  the  Student’s  t-distribution  for  n-1  degrees  of  freedom  and  ( l -a )  overall 
certainty.  A  confidence  interval  is  a  range  in  which  the  mean  of  a  population  lies  within  a  given 
certainty.  An  increase  in  samples  affects  the  width  of  the  interval,  typically  reducing  the  width. 
The  confidence  interval  in  this  test  represents  a  95  percent  probability  that  the  mean  of  the 
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LADAR  estimated  ranges  lies  within  the  interval.  Ideally,  the  confidence  interval  should 
encompass  the  actual  range.  Cases  that  showed  an  actual  range  outside  of  the  confidence 
interval  indicated  an  absolute  LADAR  range  error. 

A  prediction  interval  is  a  range  from  which  the  next  sample  will  come  within  a  given  probability. 
The  larger  the  variability  in  sample  set,  the  larger  the  prediction  interval.  The  prediction  interval 
is  typically  larger  than  the  confidence  interval.  Prediction  intervals  for  95  percent  certainty  are 
provided  for  LADAR  range  measurements  of  each  target,  implying  95  percent  probability  that 
the  next  LADAR  measurement  of  the  target  will  fall  within  the  provided  interval.  The  prediction 
interval  is  given  by 


X±tn- 1 


(a/2)s 

> 


The  LADAR  measurements  for  a  single  pixel  exhibit  a  variability  that  follows  a  normal 
distribution.  See  sample  normal  probability  plots  of  range  estimations  from  both  algorithms  in 
figures  9  and  10. 


Normal  Probability  Plot  of  1 00  ft  Target  LADAR  Data 


LADAR  Measured  Range  (ft) 


Figure  9:  Normal  probability  plot  of  30  LADAR  measurements  from 
reflective  target  “One  Hundred”  using  the  NOVAS  algorithm 
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Normal  Probability  Plot  of  1 00  ft  Target  LADAR  Data 


LADAR  Measured  Range  (ft) 


Figure  10:  Normal  probability  plot  of  30  LADAR  measurements  from 
reflective  target  “One  Hundred”  using  an  ASC-developed  ranging  algorithm 

Inspection  of  table  1  shows  that  the  actual  range  to  each  target  does  not  always  fall  inside  the 
confidence  intervals  for  the  population  mean  of  LADAR  range  measurements.  Additionally,  the 
actual  range  only  occasionally  falls  within  the  prediction  intervals  for  the  next  measured 
samples.  Confidence  and  prediction  intervals  that  contain  the  actual  target  range  are  shown 
circled  in  green  in  table  1 .  This  implied  that  a  deficiency  in  the  LADAR  equipment  and  system 
likely  existed  regardless  of  the  estimation  algorithm  used. 


Target  Codename 

Actual 

Target  Range  (ft) 

LADAR  Measured  Range  (n=30) 

95%  Confidence  Interval 

95%  Prediction  Interval 

Mean  (ft) 

Std.  Dev.  (ft) 

Lower  Limit  (ft) 

Upper  Limit  (ft) 

Lower  Limit  (ft) 

Upper  Limit  (ft) 

Fifty 

55.6 

55.3 

0.5 

55.1 

55.5 

54.2 

56.4 

E 

x: 

One  Hundred 

105.6 

105.9 

0.4 

105.8 

106.1 

105.2 

106.7 

•c 

Two  Hundred 

205.6 

205.9 

0.5 

205.7 

206.1 

204.8 

206.9 

& 

Two  Hundred  One 

206.6 

205.0 

0.7 

204.7 

205.2 

203.6 

206.4 

Two  Hundred  Four 

209.6 

207.9 

0.5 

207.7 

208.1 

206.8 

208.9 

g 

Three  Hundred 

305.6 

305.6 

0.3 

305.5 

305.7 

304.9 

306.3 

z 

Fifty 

55.6 

54.5 

0.2 

54.4 

54.5 

54.1 

54.8 

E 

One  Hundred 

105.6 

104.9 

0.3 

104.8 

105.0 

104.3 

105.5 

is 

Two  Hundred 

205.6 

206.3 

0.3 

206.2 

206.4 

205.7 

206.9 

o 

O) 

Two  Hundred  One 

206.6 

200.1 

0.7 

199.9 

200.4 

198.7 

201.6 

< 

Two  Hundred  Four 

209.6 

202.3 

0.6 

202.1 

202.6 

201.1 

203.6 

o 

0) 

Three  Hundred 

305.6 

305.7 

0.3 

305.6 

305.8 

305.1 

306.3 

< 

Table  1:  LADAR  range  measurements  and  95%  confidence  &  prediction  intervals 
compared  to  truth  values  using  the  NOVAS  and  ASC  algorithms 


Overall,  the  ASC  LADAR  proved  to  be  inaccurate.  The  overall  root  mean  squared  error 
(RMSE)  for  range  estimations  against  range  known  targets  was  8.9  inches.  However,  the  range 
estimation  error  was  dependant  on  target  range.  Figure  1 1  shows  the  RMSE  individually  for  the 
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50,  100,  200,  and  300  foot  targets.  It  could  be  seen  that  there  was  a  direct  relationship  between 
target  range  and  range  estimation  RMSE.  What  could  not  be  determined  from  the  data  collected 
was  the  nature  of  the  relationship.  The  implication  of  the  linear  fit  was  that  there  was  some 
optimized  range — in  this  case  approximately  350  feet — where  the  error  was  zero.  Beyond  this 
range  the  error  would  continue  to  grow  linearly.  More  testing  with  targets  beyond  400  feet  must 
be  accomplished  to  determine  the  relationship  between  target  range  and  range  estimation  error. 
Accomplish  testing  of  the  ASC  LADAR  with  targets  at  ranges  400  feet  and  beyond.  (R4) 

Most  importantly,  however,  was  the  fact  that  the  error  at  close  ranges  also  grew  to  unacceptable 
levels.  Estimations  held  that  the  target  F-16  would  only  be  clearly  visible  once  inside  200  feet  of 
range  where  the  range  estimation  error  was  approximately  eight  inches.  Additionally,  any  future 
application  to  aerial  refueling  of  an  unmanned  vehicle  would  suffer  greatly  from  the  decrease  in 
accuracy  that  occurred  at  short  range. 
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Figure  1 1 :  Root  mean  squared  error  of  estimated  range  to  four  ground  test  targets 


Effect  of  Timing  System  Variability  on  Range  Results 

The  most  likely  source  of  error  in  the  range  estimations  was  the  timing  system  variability  in  the 
LADAR  system.  The  oscillator  controlling  the  timing  clock  operated  at  a  nominal  400  mega- 
Hertz  (MHz)  and  thus  the  system  was  designed  to  count  time  slices  at  intervals  of  2.5 
nanoseconds  (ns)  (reference  2).  A  single  time  slice  was  therefore  equivalent  to  1 .2295  feet  of 
range.  However,  it  was  observed  that  the  oscillator  varied  in  frequency,  changing  the  time 
interval  and  therefore  changing  the  range  attributed  to  each  time  slice.  For  example,  an  oscillator 
operating  at  410  MHz  would  count  81.3  time  slices  for  a  target  at  100  feet.  If  the  ranging 
algorithm  were  calibrated  to  400  MHz,  it  would  report  these  81.3  time  slices  as  97.5  feet.  In 
addition,  from  the  time  the  initial  laser  pulse  fired,  an  unknown  time  period  on  the  order  of 
nanoseconds  passed  before  information  was  recorded  resulting  in  an  unknown  bias  of 
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approximately  8  to  12  feet.  A  calibration  was  intended  to  reduce  the  bias  error  and  correct  the 
algorithm  to  use  the  same  oscillation  frequency  of  the  camera  at  the  time  of  data  collection. 

The  NOVAS  algorithm  was  calibrated  using  the  collected  data  set.  The  first  step  was  to  find  the 
average  oscillator  frequency  of  the  camera.  A  set  of  30  measurements  to  each  of  the  targets 
placed  at  100  and  200  feet  were  used.  An  oscillator  frequency  was  calculated  to  make  the 
average  reported  relative  range  between  these  two  targets  equal  to  100  feet.  The  second  step 
was  to  remove  the  bias.  A  set  of  30  measurements  to  each  of  the  targets  at  100,  200,  and  300 
feet  were  used.  The  average  range  error  reported  by  the  algorithm  to  all  of  the  targets  was 
attributed  to  the  bias  and  removed  from  all  subsequent  measurements.  The  average  clock 
frequency  used  for  the  data  analysis  was  410  MHz,  with  an  average  offset  bias  of  5.82  feet. 

The  contractor  provided  estimation  algorithm  calibration  worked  on  the  same  principle,  but  was 
much  more  limited  in  application.  Within  this  software,  the  calibration  routine  was  based  on  a 
single  measurement  to  two  of  the  known  range  targets.  It  also  calculated  the  oscillator  frequency 
based  on  the  known  relative  range  between  the  targets  and  subsequently  removed  the  bias  based 
on  the  absolute  ranging  error  to  both  the  targets.  The  oscillator  frequency  used  by  the  contractor 
provided  algorithm  was  406  MHz  with  an  offset  bias  of  8.53  feet. 

The  limitation  of  the  ASC  calibration  arose  from  the  variability  in  range  measurements  from  the 
LADAR.  In  the  previous  section  it  was  shown  that  any  single  measurement  from  the  LADAR 
had  a  95  percent  chance  of  being  within  approximately  8  inches  of  the  actual  value.  Therefore  it 
was  reasonable  that  the  measured  relative  range  between  two  LADAR  data  points  could  have 
had  as  much  as  a  1.3  foot  error  between  them.  This  would  induce  up  to  a  1 .3  percent  ranging 
error  for  the  test  in  which  a  100  foot  relative  range  was  used  to  calibrate  the  algorithm,  but  up  to 
a  5.2  percent  error  if  the  minimum  range  of  25  feet  recommended  by  ASC  was  used  to  calibrate 
the  algorithm  (reference  3). 

Table  1  contains  evidence  that  the  NOVAS  algorithm  with  an  averaged  calibration  slightly 
outperformed  the  contractor  algorithm  with  a  single  point  calibration.  However,  the  overall  poor 
range  accuracy  could  be  attributed  to  the  fact  that  the  varying  frequency  of  the  oscillator  made 
any  calibration  invalid  for  any  LADAR  data  except  for  those  data  used  to  create  the  calibration. 
In  order  to  calibrate  the  LADAR  for  a  given  scene,  two  objects  in  that  scene  must  be  at  a  known 
range  from  the  LADAR  itself.  This  makes  this  system  inaccurate  for  an  operational  use 
requiring  range  accuracy.  Some  potential  solutions  would  have  been  to  improve  the  system 
oscillator,  add  a  system  to  monitor  the  oscillator  frequency,  or  by  use  of  fiber  optic  cables, 
provide  two  known-range  targets  to  two  of  the  pixels  on  the  imaging  array,  allowing  a  calibration 
to  be  performed  for  every  LADAR  image.  Implement  compensation  for  the  system  clock 
variability.  (R5) 

Signal  Processing  Analysis 

The  method  the  ASC  LADAR  used  to  capture  the  laser  return  pulse  waveform  affected  the 
accuracy  of  pulse  shape  correlation  algorithms  like  the  NOVAS  algorithm.  The  NOVAS 
algorithm,  like  most  matched  filter  range  estimation  algorithms,  used  correlation  techniques  to 
match  the  shape  of  the  return  pulse.  A  key  feature  of  the  NOVAS  was  to  match  the  left  and  right 
halves  of  the  return  pulse  separately.  The  logic  used  by  the  ASC  LADAR  often  truncated  the 
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right  half  of  the  return  pulse  greatly  reducing  the  advantage  the  NOVAS  may  have  over  other 
algorithms. 

The  ASC  LADAR  trigger  logic  waited  until  the  received  energy  exceeded  a  certain  threshold 
then  recorded  the  12  time  slices  prior,  the  7  time  slices  after,  and  a  single  marker  slice  at  the  end. 
The  marker  slice  was  set  to  zero  and  used  as  a  reference  for  determining  range.  Inserting  the 
marker  slice  corrupted  the  slices  just  before  and  just  after  the  marker  slice,  making  the  useable 
waveform  only  17  slices  long.  Because  the  pulse  always  triggered  on  the  leading  edge,  the 
recorded  pulse  was  usually  shifted  to  the  right  of  the  frame  and  was  often  truncated  immediately 
after  the  peak.  Often  the  right  half  of  the  pulse  was  not  captured  and  therefore  could  not  be  used 
as  part  of  the  shape  correlation.  See  appendix  B  for  a  detailed  explanation  of  this  process  and  the 
specific  effect  it  had  on  the  NOVAS  estimator  as  well  as  other  well  known  correlation  methods. 

If  the  trigger  logic  were  modified  such  that  it  recorded  fewer  frames  before  the  trigger  point  and 
more  frames  after  the  trigger  point,  more  of  the  waveform  would  be  captured.  Capturing  the 
entire  rise  and  fall  of  the  waveform  would  improve  the  ranging  accuracy  of  the  any  pulse  shape 
correlation  algorithm,  including  the  NOVAS  estimator.  Change  the  trigger  logic  of  the  ASC 

LADAR  such  that  it  captures  the  entire  rise  and  fall  of  the  return  laser  waveform.  (R6) 

System  Performance  Analysis 

The  LIL  RASCAL  was  incapable  of  accurately  ranging  a  fighter  type  aircraft  at  operationally 
representative  ranges.  The  configuration  did  not  have  the  capability  to  accurately  range  an  F-16 
target  at  300  feet  at  ground  test  conditions.  Even  with  the  aircraft  presenting  perpendicular 
surfaces  for  maximum  reflection,  the  skin  return  was  barely  above  the  level  of  the  ambient  noise. 
In  trigger  mode,  the  returns  from  the  aircraft  were  below  the  set  threshold  and  were  not  captured 
at  all  except  for  a  few  returns  from  the  slightly  more  reflective  tail  flash,  as  seen  in  figure  7. 

In  SULAR  mode,  which  has  no  threshold,  the  aircraft  was  clearly  visible  when  viewed  from  an 
intensity  standpoint,  but  only  because  of  the  uniformity  of  the  intensity  of  returns  from  the 
aircraft.  The  relatively  constant  intensity  returns  from  the  aircraft  provided  contrast  against  the 
noisy  background  even  though  the  actual  intensity  of  the  returns  is  only  slightly  greater  than  the 
background.  Notice  in  figure  12,  in  which  return  intensity  is  indicated  by  color,  that  the 
background  contains  the  same  intensity  levels  as  the  aircraft  return,  but  the  uniformity  of  the 
returns  from  the  aircraft  allow  the  image  to  be  perceived.  The  effect  this  had  on  the  ranging 
algorithm  was  that  the  aircraft  could  not  be  distinguished  from  the  background,  and  became  lost 
in  a  cloud  of  noisy  range  spikes  as  seen  in  figure  13.  This  indicated  that  reducing  the  threshold 
level  in  trigger  mode  would  not  improve  the  imaging  of  the  aircraft,  as  it  would  allow  a 
prohibitive  amount  of  noise  to  be  recorded. 
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Figure  12:  Image  showing  relative  intensity  of  recorded  laser  energy  by 
each  pixel  in  a  single  LADAR  return  captured  in  SULAR  mode 


Figure  13:  Three  dimensional  image  of  ranged  pixels  from  the  same 
LADAR  return  shown  in  figure  12 

The  maximum  range  performance  was  limited  by  the  amount  of  laser  energy  recorded  by  the 
detector.  This  could  have  been  due  to  laser  peak  power  output,  laser  attenuation  through  the 
RASCAL  lower  window,  detector  sensitivity,  attenuation  or  scattering  from  the  ground  test 
mirror,  or  LADAR  settings.  While  the  LADAR  was  set  for  nominal  operation,  there  were  ways 
to  increase  the  gain  and  sensitivity  of  the  system  that  would  increase  the  maximum  range.  The 
limited  time  available  did  not  allow  for  adjustment  to  these  settings  and  as  such  a  full  exploration 
of  the  maximum  range  was  not  accomplished.  A  complete  analysis  of  the  effect  of  these  settings 
may  result  in  a  larger  maximum  range  against  an  F-16  target.  Perform  a  ground  test  analysis 
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to  characterize  the  affect  test  conditions  and  LADAR  system  settings  on  LADAR  maximum 
range.  (R7) 

While  too  little  energy  return  limited  the  maximum  range,  too  much  energy  return  limited  the 
ability  to  accurately  range  to  the  target.  LADAR  returns  that  exceeded  approximately  3500 
digital  counts  surpassed  the  maximum  current  available  to  each  individual  detector,  resulting  in  a 
saturated  return.  The  true  peak  of  the  return  had  more  energy  than  what  could  be  recorded,  and 
the  true  shape  of  the  return  waveform  was  not  captured.  Instead  of  a  sharply  peaked  pulse  with 
well  defined  slope  on  each  side,  a  saturated  pulse  was  characterized  by  a  sharp  rise,  a  plateau¬ 
like  top,  and  a  jagged  fall  off.  See  appendix  B  for  a  comparison  between  the  return  types.  Since 
the  NOVAS  estimator  used  the  precise  range  information  carried  in  the  peak  of  the  return,  this 
saturation  limited  the  accuracy  of  range  estimation.  The  saturated  pulse  flattened  out  the  peak 
which  became  ambiguous  within  the  plateau-like  region  at  the  top.  The  true  target  location 
within  the  saturated  pulse  could  not  be  determined  and  ranging  error  was  increased  by  up  to 
several  feet.  This  became  a  problem  when  targets  were  within  50  feet  of  the  LIL  RASCAL  or 
too  highly  reflective. 

The  combined  effect  of  saturated  pulses  and  small  maximum  range  highlighted  the  limited 
window  of  range  and  reflectivity  where  the  LADAR  provided  optimal  range  information.  This 
region  where  targets  return  enough  energy  to  be  accurately  ranged  without  saturating  the  detector 
will  be  a  very  restrictive  limitation  on  future  flight  test.  Without  improvement,  system 
limitations  discovered  during  this  ground  test  would  require  targets  to  be  highly  optimized, 
restricting  flexibility.  Improve  the  characteristics  of  the  ASC  LADAR  such  that  targets  can 
be  accurately  ranged  at  longer  distances  while  close  targets  do  not  saturate  the  detector. 
(R8) 
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CONCLUSIONS  AND  RECOMMENDATIONS 

Overall,  the  laser  detection  and  ranging  (LADAR)  internally  installed  reconfigurable  airborne 
sensor,  communications  and  laser  (LIL  RASCAL)  system  did  not  achieve  expected  range 
accuracy  against  static  ground  targets  and  would  be  unsuitable  for  ranging  to  airborne  targets  in 
the  tested  configuration. 

In-cockpit  system  control  and  monitoring  was  unsatisfactory.  The  ECM  C-9492  control  panel  in 
the  F-16D  cockpit  used  to  operate  the  LADAR  system  in  the  RASCAL  pod  was  found  to  be 
unsuitable  in  ground  operations.  The  control  panel  was  not  intuitive  for  operating  the  system 
based  on  the  design  of  the  buttons  used.  The  modification  operational  supplement  (MOS) 
providing  instruction  for  operation  of  the  LIL  RASCAL  was  inaccurate  and  unclear.  The  LIL 
RASCAL  was  designed  to  present  imaging  in  real  time  to  the  operator  in  the  cockpit  through  the 
cockpit  multi-function  display  (MFD),  but  this  feature  was  not  demonstrated  in  the  ground  test 
due  to  system  integration  issues. 

The  LADAR  did  not  achieve  expected  range  accuracy.  Confidence  intervals  and  prediction 
intervals  for  individual  target  ranges  computed  from  the  ground  test  results  rarely  encompassed 
the  actual  truth  range  to  the  target.  The  total  root  mean  squared  error  (RMSE)  of  range 
measurements  from  the  LADAR  was  8.9  inches.  However,  RMSE  was  dependant  on  range  with 
a  worst  case  RMSE  of  13.9  inches  at  50  feet  and  insufficient  data  to  predict  performance  outside 
of  300  feet. 

Timing  system  variability  of  the  LADAR  system  was  unsatisfactory.  Two  forms  of  calibration 
were  used  to  compensate  for  the  variability:  a  single  point  calibration  employed  by  the 
contractor  provided  software,  and  a  30  point  calibration  employed  by  the  normalized  variable 
shaping  (NOVAS)  estimator  developed  by  the  test  team.  Both  calibrations  attempted  to 
overcome  variability  in  the  system  oscillator  frequency  which  ultimately  determined  the  target 
range.  Neither  method  was  able  to  compensate  for  the  fact  that  essentially  every  image  needed  a 
separate  calibration,  requiring  two  targets  of  known  range  to  be  present  in  the  image. 

The  method  the  ASC  LADAR  used  to  capture  the  laser  return  pulse  waveform  affected  the 
accuracy  of  pulse  shape  correlation  algorithms  like  the  NOVAS  algorithm.  A  key  feature  of 
NOVAS  was  to  match  the  left  and  right  halves  of  the  return  pulse  separately.  The  logic  used  by 
the  ASC  LADAR  often  truncated  the  right  half  of  the  return  pulse  greatly  reducing  the  advantage 
NOVAS  may  have  over  other  algorithms.  If  the  trigger  logic  were  modified  such  that  it  recorded 
fewer  frames  before  the  trigger  point  and  more  frames  after  the  trigger  point,  more  of  the 
waveform  would  be  captured.  Capturing  the  entire  rise  and  fall  of  the  waveform  would  improve 
the  ranging  accuracy  of  pulse  shape  correlation  algorithms  like  NOVAS. 

The  configuration  did  not  have  the  capability  to  generate  a  clear,  ranged  image  of  an  F- 16  at  300 
feet  during  the  ground  test.  Even  with  the  aircraft  presenting  perpendicular  surfaces  for 
maximum  reflection,  the  skin  return  was  barely  above  the  level  of  the  ambient  noise.  In  trigger 
mode,  the  returns  from  the  aircraft  were  below  the  set  threshold  and  were  not  captured  at  all 
except  for  a  few  returns  from  the  slightly  more  reflective  tail  flash.  In  SULAR  mode,  which  has 
no  threshold,  the  aircraft  was  visible  when  viewed  from  an  intensity  standpoint,  but  only  because 
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of  the  uniformity  of  the  intensity  of  returns  from  the  aircraft.  It  was  not  possible  to  separate 
range  information  from  the  background  noise. 

The  maximum  range  performance  was  limited  by  the  amount  of  laser  energy  recorded  by  the 
detector.  This  could  have  been  due  to  laser  output,  laser  attenuation  through  the  RASCAL  lower 
window,  detector  sensitivity,  attenuation  or  scattering  from  the  ground  test  mirror,  or  LADAR 
settings.  While  too  little  energy  return  limited  the  maximum  range,  too  much  energy  return 
limited  the  ability  to  accurately  range  to  the  target.  Since  the  NOVAS  estimator  used  the  precise 
range  information  carried  in  the  peak  of  the  return,  this  saturation  limited  the  accuracy  of  range 
estimation.  True  target  location  within  a  saturated  pulse  could  not  be  determined  and  ranging 
error  was  increased  by  up  to  several  feet.  The  combined  effect  of  saturated  pulses  and  small 
maximum  range  highlighted  the  limited  window  of  range  and  reflectivity  where  the  LADAR 
provided  optimal  range  information.  This  region  where  targets  return  enough  energy  to  be 
accurately  ranged  without  saturating  the  detector  will  be  a  very  restrictive  limitation  on  future 
flight  test. 

The  above  conclusions  led  the  Walker  Ranger  team  to  make  the  following  recommendations  in 
priority  order: 

•  Provide  video  from  the  LIL  RASCAL  to  the  F-16  MFD  to  allow  assessment  in  real 
time  of  LADAR  operation.  (R2,  page  8) 

•  Perform  a  ground  test  analysis  to  characterize  the  affect  test  conditions  and  LADAR 
system  settings  on  LADAR  maximum  range.  (R7,  page  17) 

•  Verify  mean  and  standard  deviation  of  GAINR-LITE  meet  predictions  from  412 
TW/ENR  prior  to  flight  test  of  the  LIL  RASCAL.  (Rl,  page  6) 

•  Implement  compensation  for  the  system  clock  variability.  (R5,  page  14) 

•  Improve  the  characteristics  of  the  ASC  LADAR  such  that  targets  can  be  accurately 
ranged  at  longer  distances  while  close  targets  do  not  saturate  the  detector.  (R8,  page 
17) 

•  Change  the  trigger  logic  of  the  ASC  LADAR  such  that  it  captures  the  entire  rise  and 
fall  of  the  return  laser  waveform.  (R6,  page  15) 

•  Provide  an  alternate  interface  or  control  panel  and  accurate  written  instructions  to 
operate  the  LADAR  system  in  the  RASCAL  pod.  (R3,  page  9) 

•  Accomplish  testing  of  the  ASC  LADAR  with  targets  at  ranges  400  feet  and  beyond. 
(R4,  page  13) 
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Figure  A-l:  Typical  SULAR  mode  images  gathered  during  the  ground  test 
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Figure  A-2:  Typical  trigger  mode  images  gathered  during  the  ground  test 
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APPENDIX  B  -  PULSE  SHAPE  EFFECT  ON  SHAPE  CORRELATION  ESTIMATION 

The  Advanced  Scientific  Concepts  (ASC)  laser  detection  and  ranging  (LADAR)  had  the 
capability  to  record  20  laser  return  intensity  values  from  each  detector  in  the  128  x  128  detector 
array.  In  a  given  instant,  the  photons  of  laser  light  falling  upon  each  single  detector  were  output 
as  a  voltage  and  stored  temporarily  in  one  of  20  locations  (referred  to  in  this  report  as  “slices”), 
allotted  to  that  detector.  Each  of  the  20  slices  were  filled  sequentially,  with  the  sequencing  being 
controlled  by  an  electronic  oscillator  operating  at  a  nominal  frequency  of  400  mega-Hertz 
(MHz).  As  time  passed,  each  detector  output  values  that  cycled  through  the  slices,  overwrote 
values  that  occurred  20  slices  ago  but  kept  track  of  the  total  number  of  slices  recorded.  At  any 
given  time,  there  was  a  20  slice  time  history  stored  that  was  updated  approximately  every  2.4 
nanoseconds. 

The  LADAR  stopped  overwriting  previous  data  based  on  the  mode  of  operation.  In  stop-mode,  a 
particular  pixel  stopped  overwriting  when  the  detector  for  that  pixel  output  a  voltage  above  a 
pre-defined  threshold.  When  this  threshold  was  exceeded  in  a  given  slice,  that  slice,  the  previous 
12  slices  and  the  subsequent  six  slices  were  saved.  The  thirteenth  previous  slice  was  set  to  zero 
value  and  was  called  the  marker  slice.  The  LADAR  permanently  stored  the  twenty  slices  as  well 
as  the  total  number  slices  (called  the  “hit  buffer”),  that  had  passed  at  the  time  the  marker  slice 
was  saved.  The  hit  buffer  could  be  related  to  the  range  to  the  marker  slice  through  the  speed  of 
light.  Slices  were  saved  in  their  original  stored  position  in  the  sequence  of  20  and  almost  always 
wrapped  around  circularly  around  the  sequence. 

Figure  B-l  shows  a  typical  LADAR  return  from  the  ground  test  as  it  was  recorded  by  the  system. 
First,  the  LADAR  stored  slices  12-20  and  1  -  3  as  they  occurred,  sequentially  overwriting  the 
previous  values.  Then  slice  four  exceeded  the  threshold  value  causing  the  system  to  trigger. 
Slice  1 1  was  set  to  zero  and  the  hit  buffer  for  slice  1 1  was  saved.  Slices  12-20  and  1-3  were 
saved  preventing  them  from  being  overwritten.  Finally,  slices  5-10  were  saved  sequentially  as 
they  occurred. 
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Figure  B-l :  Typical  unaltered  LADAR  return 
recorded  during  the  ground  test 

In  staring  underwater  laser  ranging  (SULAR)  mode  all  pixels  stopped  overwriting 
simultaneously  after  a  pre-determined  amount  of  time  had  passed  after  the  emission  of  the 
outgoing  pulse.  The  format  of  the  recorded  data  was  identical  to  that  shown  in  figure  B-l, 
however  the  marker  slice  and  hence  the  19  slices  leading  up  to  it  were  placed  based  on  time  and 
not  triggered  off  of  a  set  threshold. 


The  first  step  to  analyzing  the  return  waveforms  by  the  normalized  variable  shaping  (NOVAS) 
range  estimation  algorithm  was  to  shift  the  points  circularly  such  that  the  marker  slice  became 
the  first  slice  and  the  remaining  19  slices  appeared  in  chronological  order.  Next,  the  slices  saved 
on  either  side  of  the  marker  slice  were  thrown  out  as  they  were  corrupted  by  an  anomaly  in  the 
LADAR  system  as  a  side-effect  of  creating  the  marker  slice.  Figure  B-2  shows  several 
representative  waveforms  gathered  during  the  ground  test  that  have  been  shifted  in  this  manner 
and  have  the  corrupted  data  points  marked.  All  three  waveforms  in  figure  B-2  triggered  in  slice 
14.  The  minimum  intensity  pulse  triggered  at  the  peak  such  that  the  six  subsequent  slices 
captured  the  entire  pulse  shape.  The  maximum  intensity  pulse  triggered  very  early  in  the  rise  of 
the  pulse  such  that  the  trailing  edge  of  the  pulse  was  not  recorded.  This  was  a  factor  for  ranging 
algorithms  like  NOVAS  that  used  pulse  shape  correlation  to  determine  the  target  range. 

The  NOVAS  algorithm  modeled  the  return  LADAR  pulse  as  a  piecewise  continuous  joining  of 
two  Gaussian  shaped  pulses  of  different  widths.  The  left  half  from  negative  infinity  to  the  pulse 
peak  was  a  Gaussian  pulse  of  one  width,  while  the  right  half  from  the  pulse  peak  to  positive 
infinity  was  a  Gaussian  pulse  of  another  width.  The  NOVAS  then  attempted  to  match  the 
position  of  the  peak  as  well  as  the  left  and  right  pulse  widths  to  find  the  combination  with  the 
highest  correlation.  The  resulting  peak  location  within  a  given  frame  of  20  slices,  plus  the  total 
range  to  the  marker  slice  range  would  then  become  the  NOVAS  estimation  for  the  target  range. 
The  NOVAS  range  estimation  was  generated  under  the  assumption  that  the  peak  of  the  true 
LADAR  return  pulse  marked  the  target  range. 
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Figure  B-2:  Typical  LADAR  returns  recorded 
during  the  ground  test,  circularly  shifted 

The  goal  of  the  NOVAS  algorithm  was  to  produce  an  estimation  of  the  true  LADAR  return  pulse 
while  reducing  the  effect  of  random  noise  in  the  signal.  The  received  signal  was  subject  to 
several  noise  sources  including  the  photonic  effects  of  shot  noise  and  Poisson  noise,  as  well  as 
system  effects  of  thermal  noise  and  quantization  error  during  an  analog  to  digital  conversion.  To 
eliminate  the  effect  of  any  bias  in  the  signal  and  to  generate  consistently  sized  NOVAS 
estimations,  LADAR  return  pulses  were  first  normalized  by  subtracting  the  remaining  1 7  slices 
from  their  mean  and  dividing  by  their  standard  deviation. 

The  mechanism  by  which  the  LADAR  stored  the  waveforms  limited  the  utility  of  a  shape 
correlating  range  estimator  by  truncating  the  trailing  half  of  the  return  pulse  for  all  returns  except 
those  just  greater  than  threshold.  Figure  B-3  shows  the  same  three  waveforms  after  they  were 
normalized  as  well  as  the  NOVAS  estimation  for  each  of  the  pulse  shapes.  It  can  be  seen  that  for 
the  minimal  intensity  return,  both  the  left  and  right  halves  of  the  pulse  had  numerous  values  upon 
which  to  perform  the  correlation,  resulting  in  a  more  accurate  solution.  The  medium  intensity 
return  had  only  three  points  on  the  right  half  of  the  curve,  while  the  maximum  intensity  return 
had  only  two  points  on  the  right  half  of  the  curve. 

The  fewer  number  of  points  made  the  shape  matching  very  susceptible  to  variation  due  to  noise 
in  the  signal.  For  example,  shifting  the  far  right  point  on  the  maximum  intensity  curve  would 
cause  a  significant  change  in  the  shape  of  the  right  half  of  the  matched  pulse  and  a  shift  in  the 
matched  peak.  Shifting  a  single  point  on  the  minimum  intensity  return  had  a  lesser  effect  on  the 
pulse  shape  and  peak  location  due  to  the  larger  number  of  points  acting  to  reduce  the  effect  of 
random  noise.  Waveforms  that  captured  the  entire  left  and  right  halves  provided  the  best 
conditions  for  countering  the  effect  of  noise,  generating  an  accurate  estimation  of  the  true  pulse, 
and  providing  an  accurate  range  estimation. 
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Figure  B-3:  Normalized  LADAR  returns  gathered  during  the 
ground  test  with  NOVAS  pulse  shaped  estimations 


Modifying  the  trigger  logic  such  that  it  saves  more  slices  after  the  trigger  point  and  fewer  prior  to 
the  trigger  point  would  alleviate  this  problem.  Swapping  the  methodology  such  that  only  six 
slices  prior  to  trigger  and  twelve  slices  after  were  saved  would  ensure  that  even  the  largest  pulses 
were  captured  in  their  entirety.  Another  option  would  be  to  raise  the  threshold  such  that  large 
pulses  trigger  later  on  the  rising  side,  extending  the  six  subsequent  frames  nearer  to  the  trailing 
edge  of  the  pulse.  However,  this  would  decrease  the  overall  sensitivity  and  only  gain  at  most 
two  additional  slices  on  the  back  side  of  the  pulse. 

Detector  saturation  also  reduced  the  accuracy  of  the  NOVAS  estimator  or  any  other  shape 
correlator.  Each  pixel  in  the  detector  array  was  unable  to  accurately  report  return  values  greater 
than  approximately  2,900  digital  counts.  Instead  the  pixel  would  report  some  maximum  level 
with  a  significant  amount  of  system  induced  variability.  This  would  deform  the  peak  of  the 
pulse  into  a  jagged  plateau,  eliminating  any  useful  information  about  the  position  of  the  true 
peak.  In  most  cases  the  saturated  waveforms  captured  by  the  LADAR  did  not  include  any 
information  past  the  plateau  due  to  the  triggering  mechanism  already  addressed.  Figure  B-4 
shows  an  example  of  such  a  saturated  waveform  with  the  NOVAS  estimation  of  the  pulse.  It  can 
be  seen  that  the  NOVAS  model  did  not  fit  well  with  the  saturated  pulse,  and  that  the  peak  of  the 
pulse  had  been  spread  across  a  plateau  that  covers  four  slices,  or  approximately  five  feet. 
Saturated  pulses  indicated  range  errors  of  up  to  two  feet  less  than  actual  target  range. 
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Normalized  Saturated  LADAR  Return  with  NOVAS  Estimation 


Figure  B-4:  Normalized  saturated  LADAR  return  gathered  during 
the  ground  test,  with  NOVAS  estimation 
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APPENDIX  C  -  LIST  OF  ABBREVIATIONS  AND  SYMBOLS 

Abbreviation  Definition 


AAR 

AFB 

AFFTC 

AFIT 

ASC 

DGPS 

EMI/C 

FLVC 

FOV 

GAINR-LITE 

GPS 

LADAR 

LIL  RASCAL 

MFD 

MHz 

mJ 

mm 

MOS 

NOVAS 

ns 

RASCAL 

RMSE 

SULAR 

TIM 

TMP 

TPS 

TV 

TW 

pm 

UAV 

USAF 


air-to-air  refueling 

Air  Force  Base 

Air  Force  Flight  Test  Center 

Air  Force  Institute  of  Technology 

Advanced  Scientific  Concepts 

differential  global  positioning  system 

electro-magnetic  interference  and  compatibility 

FLASH  LADAR  Video  Camera 

field  of  view 

GPS  aided  internal  navigation  reference 

global  positioning  system 

laser  detection  and  ranging 

LADAR  Internally  Loaded  RASCAL 

multi-function  display 

mega-Hertz 

milli-Joule 

millimeter 

modification  operational  supplement 
normalized  variable  shaping 
nanoseconds 

Reconfigurable  Airborne  Sensor,  Communications  and  Laser 

root  mean  squared  error 

staring  underwater  laser  ranging 

technical  information  memorandum 

test  management  project 

Test  Pilot  School 

television 

Test  Wing 

micrometer 

unmanned  aerial  vehicle 
United  States  Air  Force 
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